Correcting Decalibration of Stereo Cameras in Self-Driving Vehicles

We address the problem of optical decalibration in mobile stereo camera setups, especially in context of autonomous vehicles. In real world conditions, an optical system is subject to various sources of anticipated and unanticipated mechanical stress (vibration, rough handling, collisions). Mechanical stress changes the geometry between the cameras that make up the stereo pair, and as a consequence, the pre-calculated epipolar geometry is no longer valid. Our method is based on optimization of camera geometry parameters and plugs directly into the output of the stereo matching algorithm. Therefore, it is able to recover calibration parameters on image pairs obtained from a decalibrated stereo system with minimal use of additional computing resources. The number of successfully recovered depth pixels is used as an objective function, which we aim to maximize. Our simulation confirms that the method can run constantly in parallel to stereo estimation and thus help keep the system calibrated in real time. Results confirm that the method is able to recalibrate all the parameters except for the baseline distance, which scales the absolute depth readings. However, that scaling factor could be uniquely determined using any kind of absolute range finding methods (e.g. a single beam time-of-flight sensor).Comment: 8 pages, 9 figure

Object tracking by a generalized Hough transform

Visual object tracking is a very diverse and useful area of computer vision. There are many different approaches to solving this problem and the goal of the thesis is to first present some of the methods that are used for implementation of state-of-the-art tracking algorithms. Secondly, the analysis of a concrete algorithm that tracks the object by using generalized Hough transform and lastly to design and implement some enhancements that boost the algorithm's robustness and accuracy. The proposed enhancements are based on Harris corner detection, Kalman filter and a segmentation algorithm that uses Markov random fields. The result of the thesis is thus an improved algorithm, implemented in C++ with added methods that improve its performance. Practical experiments were carried out in a framework designed for testing tracking algorithms by using diverse and difficult video sequences. Experiment results clearly show the improvements caused by the proposed methods

Object tracking by a generalized Hough transform

Visual object tracking is a very diverse and useful area of computer vision. There are many different approaches to solving this problem and the goal of the thesis is to first present some of the methods that are used for implementation of state-of-the-art tracking algorithms. Secondly, the analysis of a concrete algorithm that tracks the object by using generalized Hough transform and lastly to design and implement some enhancements that boost the algorithm's robustness and accuracy. The proposed enhancements are based on Harris corner detection, Kalman filter and a segmentation algorithm that uses Markov random fields. The result of the thesis is thus an improved algorithm, implemented in C++ with added methods that improve its performance. Practical experiments were carried out in a framework designed for testing tracking algorithms by using diverse and difficult video sequences. Experiment results clearly show the improvements caused by the proposed methods

Visual object tracking with a quadcopter

Because of increasing accessibility and quality of small aerial robots for personal use we wanted to upgrade their functionality with the ability to autonomously follow objects. This can be achieved by using tracking algorithms that work with visual information. Such algorithms locate the object in every image and can adapt to size and appearance changes of the object. We have formulated a system that uses information from a tracking algorithm to control the quadcopter. The proposed system is thus able to maintain the distance to the object as well as keep the object near the center of the image. By using a platform with a movable camera we have achieved high responsiveness as well as the ability to follow an object that is located lower than the quadcopter itself. Our system also manages to follow targets that move on non-planar surfaces which is not possible using a fixed camera

Object tracking by a generalized Hough transform

Sledenje objektov je zelo raznoliko in uporabno področje računalniškega vida. Pristopov k reševanju tega problema je ogromno, namen tega dela pa je predstaviti nekaj metod, ki se uporabljajo za implementacijo naprednih sledilnih algoritmov, analizirati konkreten algoritem, ki sledenje izvaja z uporabo generaliziranega Houghovega transforma ter načrtati in izvesti izboljšave, ki bodo koristile algoritmovi robustnosti in natančnosti. Predlagane izboljšave temeljijo na uporabi Harrisove detekcije oglišč, Kalmanovega filtra in segmentacijskega algoritma, ki deluje na podlagi Markovovih slučajnih polj. Rezultat diplomskega dela je tako izboljšan algoritem, implementiran v C++, z dodanimi metodami, ki mu omogočajo boljše delovanje. Praktični eksperimenti so bili izvedeni v okolju, namenjemu testiranju sledilnih algoritmov, z uporabo raznolikih in zahtevnih video sekvenc. Rezultati eksperimentov jasno prikazujejo izboljšave, ki so jih povzročile predlagane metode.Visual object tracking is a very diverse and useful area of computer vision. There are many different approaches to solving this problem and the goal of the thesis is to first present some of the methods that are used for implementation of state-of-the-art tracking algorithms. Secondly, the analysis of a concrete algorithm that tracks the object by using generalized Hough transform and lastly to design and implement some enhancements that boost the algorithm\u27s robustness and accuracy. The proposed enhancements are based on Harris corner detection, Kalman filter and a segmentation algorithm that uses Markov random fields. The result of the thesis is thus an improved algorithm, implemented in C++ with added methods that improve its performance. Practical experiments were carried out in a framework designed for testing tracking algorithms by using diverse and difficult video sequences. Experiment results clearly show the improvements caused by the proposed methods

Object tracking by a generalized Hough transform

Sledenje objektov je zelo raznoliko in uporabno področje računalniškega vida. Pristopov k reševanju tega problema je ogromno, namen tega dela pa je predstaviti nekaj metod, ki se uporabljajo za implementacijo naprednih sledilnih algoritmov, analizirati konkreten algoritem, ki sledenje izvaja z uporabo generaliziranega Houghovega transforma ter načrtati in izvesti izboljšave, ki bodo koristile algoritmovi robustnosti in natančnosti. Predlagane izboljšave temeljijo na uporabi Harrisove detekcije oglišč, Kalmanovega filtra in segmentacijskega algoritma, ki deluje na podlagi Markovovih slučajnih polj. Rezultat diplomskega dela je tako izboljšan algoritem, implementiran v C++, z dodanimi metodami, ki mu omogočajo boljše delovanje. Praktični eksperimenti so bili izvedeni v okolju, namenjemu testiranju sledilnih algoritmov, z uporabo raznolikih in zahtevnih video sekvenc. Rezultati eksperimentov jasno prikazujejo izboljšave, ki so jih povzročile predlagane metode.Visual object tracking is a very diverse and useful area of computer vision. There are many different approaches to solving this problem and the goal of the thesis is to first present some of the methods that are used for implementation of state-of-the-art tracking algorithms. Secondly, the analysis of a concrete algorithm that tracks the object by using generalized Hough transform and lastly to design and implement some enhancements that boost the algorithm\u27s robustness and accuracy. The proposed enhancements are based on Harris corner detection, Kalman filter and a segmentation algorithm that uses Markov random fields. The result of the thesis is thus an improved algorithm, implemented in C++ with added methods that improve its performance. Practical experiments were carried out in a framework designed for testing tracking algorithms by using diverse and difficult video sequences. Experiment results clearly show the improvements caused by the proposed methods

Joint Calibration of a Multimodal Sensor System for Autonomous Vehicles

Multimodal sensor systems require precise calibration if they are to be used in the field. Due to the difficulty of obtaining the corresponding features from different modalities, the calibration of such systems is an open problem. We present a systematic approach for calibrating a set of cameras with different modalities (RGB, thermal, polarization, and dual-spectrum near infrared) with regard to a LiDAR sensor using a planar calibration target. Firstly, a method for calibrating a single camera with regard to the LiDAR sensor is proposed. The method is usable with any modality, as long as the calibration pattern is detected. A methodology for establishing a parallax-aware pixel mapping between different camera modalities is then presented. Such a mapping can then be used to transfer annotations, features, and results between highly differing camera modalities to facilitate feature extraction and deep detection and segmentation methods

Correcting decalibration of stereo cameras in self-driving vehicles

Camera systems in autonomous vehicles are subject to various sources of anticipated and unanticipated mechanical stress (vibration, rough handling, collisions) in real-world conditions. Even moderate changes in camera geometry due to mechanical stress decalibrate multi-camera systems and corrupt downstream applications like depth perception. We propose an on-the-fly stereo recalibration method applicable in real-world autonomous vehicles. The method is comprised of two parts. First, in optimization step, external camera parameters are optimized with the goal to maximise the amount of recovered depth pixels. In the second step, external sensor is used to adjust the scaling of the optimized camera model. The method is lightweight and fast enough to run in parallel with stereo estimation, thus allowing an on-the-fly recalibration. Our extensive experimental analysis shows that our method achieves stereo reconstruction better or on par with manual calibration. If our method is used on a sequence of images, the quality of calibration can be improved even further